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Who are we? 



Professor: 

Aaron Bobick 

afb@cc.aatech.edu 

Office: CCB 316 
Office hours: 

Tues l-2pm (email is 
better) 


TA: 

Ana Huaman 

ahuaman3@aatech.edu 

Office: CCB 2xx 
Office hours: TBD 

Yin Li 

happyharrvcn@gmail.com 

Office: CCB 308 
Office hours: TBD 


much 
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Where are you? 

• CS 4495 Computer Vision 


• Web site: http://www.cc.qatech.edu/~afb/classes/CS4495-Fall2014/ 

• Will have posted calendar/syllabus with posted slides, problem sets with 
data, other administrative stuff. 

• T-square: The usual stuff. There is/will be a web page under 
resources that points to the class web page listed above. 

• Slides: PDFs will be posted by linking to the calendar. Hopefully draft 
by class time. 

• Piazza: You will all receive an invitation to Piazza for CS4495. If not, 
send us email. This was invaluable for discussions for problem sets. 

• Announcements will likely be done through both T-Square (email) and Piazza 

• Matlab access: if you don’t know how to get Matlab access, first ask 
a friend. Then come see TAs or me. And ask about Python/OpenCV 
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Introduction 


Where are you? 

•CS 7495 Computer Vision 
•All of the above plus... 

• Web site: 

http://www.cc.qatech.edu/~afb/classes/CS7495- 

Fall2014/ 


• It is not there yet! 
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Who are you? - CS4495 

• From the web site: 

This is a senior level undergraduate course for those interested in 
computer vision. The GR section is open to graduate students who 
are MSCS students or who need a solid undergraduate grounding 
before /while taking the graduate version (CS7495). 

• What do you know? 

• Data structures - you’ll be writing code that builds representations of 
images, features, and geometric constructions 

• Matlab (or Python with OpenCV or if really desparate C++/OpenCV) 

• More math than most CS courses: Linear algebra, Vector Calculus, 
Linear algebra, Probability, Linear algebra 

• No CV assumed but maybe you’ve hacked an image? 
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Who are you? - CS7495 

• From the (soon to be) web site: 

The graduate class in computer vision that assumes a general 
background at the undergraduate level. 

• What do you know? 

• Everything before plus basic computer vision 
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What will you read? 


Forsythe and Ponce: 


Computer Vision: A 
Modem Approach (2 nd 
Ed) 



Computer Vision 

A MODERN APPROACH 


(no pdf) 


Introduction 


Rick Szeliski’s 
book: 


Computer Vision: 
Algorithms and 
Applications 


TIHTS IN COMPUTE* StliHCS 

Computer Vision 

Algorithms and Applications 



Richard Szell ski 


Springer 
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What you will do? - CS4495 

• The grade is mostly problem set based (85%). Currently 

8 tentatively scheduled: 

0 Due Thurs Aug 29th midnight. *Almost already posted* This is 
just to make sure you can read, numerically manipulate, and 
display images. This one is no work. Others are serious 
work. 

1. Edges, lines, Hough transform 

2. Stereo Matching 

3. Calibration - where is the camera given the points. 

4. Relative motion - SIFT features to find where the cameras are 

5. Motion flow - or maybe Pyramid match kernel???? 

6. Tracking 

7. Motion History Images (action recognition) 

• And there will be an exam - 15% 
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What you will do? - CS7495 

• There will be a paper assigned per week of more current 
or advanced material. 

• There will be an optional one hour meeting each week to 
discuss the paper (or projects). 

• There will be a simple summary question for each paper that you’ll 
hand in. Easy to answer if you attend the session. 

• The grade is mostly problem on three projects. 

• See previous year’s web site: 

• http://www.cc.qatech.edu/~dellaert/13F- 

7495/Svllabus.html 
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When will you do them? - CS4495 

• Except for PSO 0, they will all be due on Sunday at 
11:59pm. 

• Late submissions will only be accepted at full credit with 
prior approval. Otherwise 50% (yes half) reduction. 

• TA/Prof *not* obligated to get back to you about 
permissionthe weekend it is due! At your own risk. 

• I will be very adamant about this. Not fair to others or the 
TAs. 
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When will you do them? - CS7495 

• Summary questions will be weekly. 

• There will be 6 deadlines for the projects - each has a 
proposal and a final writeup. 
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With whom will you do it? - CS4495 

• Honesty/Integrity policy (from web site): 

Problem sets are to be done individually but you may collaborate 
at the “white board level” helping each other with algorithms and 
general computation, BUT YOUR CODE MUST BE YOUR OWN. 


• Do not hand in other people’s code unless you (1) say you 
are, and (2) you want no credit for that section. We will be 
explicit about what previous or provided code you can 
use. 

• Grad students (CS4495GR) will be graded as a separate 
pool so as to not penalize undergraduates. (Some day it 
will be CS6495.) Otherwise, deliverables are identical. 
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How will you do it? - CS4495 

• We will mostly support Matlab/Octave 

• In class, most of the time we will use Matlab to handle the image 
processing (though we’ll learn some of that) and then your data 
structures for the vision. 

• If you want be even more “job ready” 

• OpenCV is a Python (and C/C++) package for computer vision (Linux 
or Windows) 

• Can be painful to install/use because OpenCV breaks with every 
version. 

• Grads: Other graduate students in RIM (Robotics and Intelligent 
Machines) can help. 

• TAs will not be obligated to help with OpenCV “issues” but we will 
make sure problem sets are Python-doable. 

• One prior student” “Don't let them use OpenCV. Too hard to do the 
extra stuff like graphs etc. “ 



CS 4495 Computer Vision - A. Bobick 


Introduction 


With whom will you do it? - CS7495 

• Summaries on your own (duh). 

• Each project must be done by 2 or 3 people. No more, no 
less. If you can not find a partner we’ll help you pair up. 
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Any questions so far 


Introduction 
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Who am I... 

• Background: degrees in Math, CS but PhD in 
CogSci/AI; interest in high level perception and 
cognition 

• Faculty at the MIT Media Lab for a while - developed 
lots of work in machine understanding of action from 
video 

• Came here 15 years ago, and collected hats: 

• Was GVU director 

• Founding chair of the School of Interactive Computing 

• But the most fun is being professor in Computational 
Perception and Robotics! 
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Research I used to do... 

Action recognition from 

video 

•Lots of domains/levels of 
complexity: 

•Body motions 
•Gesture recognition, 
•Football plays 
•Aware Home 
•Surveillance 


Introduction 
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now.. 



Robots that “see”... not (just) a question of geometry 

but understanding. 


i 1 


*'* ** 




t- 1 


ROBOT (SIMON wavi 


^ External 
Camera 




Indoor 


ING WITH BLOCKS 


Outdoor 


Human Robot Interaction 
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Affordance-based 


perception: Robot learning 
the “affordances” of objects 
and how to use in planning 

and acting. 


Two specific projects: 


Vision for) Human-robot 
collaboration in manufacturing 

(with BMW!) 
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Why study Computer Vision? 

• Images (and movies) have become ubiquitous in both 
production and consumption. 

• Therefore application to manipulate images (movies) are 
becoming core. 

• As are systems that extract information from imagery 

• Surveillance 

• Building 3D representations 

• Motion capture assisted 

• But most of all... 


It is a really deep and cool set of problems! 
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What is computer vision? 



Terminator 2 


Steve Seitz 
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• Goal of computer vision is to write computer programs that can 
interpret images 


Steve Seitz 
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Can computers match (or beat) human vision? 



¥'■ 







Yes and no (but mostly no!) 


• humans are much better at “hard” things 

• computers can be better at “easy” things 


Steve Seitz 
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Human perception has its shortcomings... 



Sinha and Poqqio, Nature. 1996 



Steve Seitz 
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Sinha and Poqqio, Nature. 1996 







Copyright A.Kitaoka 2003 


Steve Seitz 
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Current state of the art 

• The next slides show some examples of what current 
vision systems can do 
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Image from Microsoft’s Virtual Earth 
(see also: Google Earth) 


Steve Seitz 
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UliK. r 



Microsoft L-w Lsi# 

Photosynth 


Home 


» Try it 

* What is Photosynth? 

* Collections 

* Team blog 

* Videos 

* System requirements 

* About us 

* FAQ 



The Photosynth Technology Preview is a taste of the newest - and, we hope, most 
exciting - way to view photos on a computer. Our software takes a large collection of 
photos of a place or an object, analyzes them for similarities, and then displays the photos in 
a reconstructed three-dimensional space, showing you how each one relates to the next. 


http://labs.live.com/photosvnth/ 


Based on Photo Tourism technology developed here in UW. 

by Noah Snavely, Steve Seitz, and Rick Szeliski 


Steve Seitz 
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Optical character recognition (OCR) 

Technology to convert scanned docs to text 

• If you have a scanner, it probably came with OCR software 







AT&T 

TiTj77 

n 

?. Net 5 

HE&EAPFCM 

answer: 

0 


0 










Digit recognition, AT&T labs 
http://www.research.att.com/~vann / 


License plate readers 

http://en.wikipedia.org/wiki/Automatic number plate recognition 


Steve Seitz 
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Face detection 



• Many new digital cameras now detect faces 

• Canon, Sony, Fuji, ... 


Steve Seitz 
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Smile detection? 


The Smile Shutter flow 

Imagine a camera smart enough to catch every smile! In Smile Shutter Mode your Cyber-shot'* 
camera can automatically trip the shutter at just the right instant to catch the perfect expression 



Smile On pi ure d ! 


S mil e Captu red ! 


Sm ile Captured! 


Sony Cyber-shot® T70 Digital Still Camera 


Steve Seitz 







CS 4495 Computer Vision - A. Bobick 


Introduction 


Face recognition 



VOL 167, NOr 6 s. 


JUNE19&5 


NATIONAL 

GEOGRAPHIC 


GREAT SALT LAKE: 

THE FLOODING 
DESERT m 

U.S.- MEXICAN BORDER: 

UFE ON TOE LINE 

JAVA'S WILDLIFE RETURNS m 

Along Afghanistan's 
War torn Frontier ™ 


FAIfl SKIES FOR THE CAYM AH ISLANDS ™ 




Hk : :nlnl fjuT Eril Lrf-itn 
■T_ri^7Pr'iiJrT J > 


SEE NATIONAL GEOGRAPHIC EXFLQRER EVERT SUN DAT CN NICKELODEON CABLE TU 


Who is she? 


Steve Seitz 
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Vision-based biometrics 



“ How the Afghan Girl i/i/as Identified by Her Iris Patterns" Read the story 
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Fingerprint scanners on 
many new laptops, 
other devices 


Face recognition systems now 

beginning to appear more widely 
http://www.sensiblevision.com/ 


Steve Seitz 
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LaneHawk by EvolutionRobotics 

“A smart camera is flush-mounted in the checkout lane, continuously watching 
for items. When an item is detected and recognized, the cashier verifies the 
quantity of items that were found under the basket, and continues to close the 
transaction. The item can remain under the basket, and with LaneHawk, you are 
assured to get paid for it. . . “ 
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Object recognition (in mobile phones!) 



• This is becoming real: 

• Google Goggles 

• Lincoln Microsoft Research 

• Point & Find, Nokia 


Steve Seitz 
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Special effects: shape capture 



The Matrix movies, ESC Entertainment, XYZRGB, NRC 


Steve Seitz 
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Special effects: motion capture 




Pirates of the Carribean, Industrial Light and Magic 

Click here for web site 


Steve Seitz 
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Sportvision first down line 
Nice explanation on www.howstuffworks.com 


Steve Seitz 
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Slide content courtesy of Amnon Shashua 

Smart cars 



News 


f nooneye Advanced 
Technologies Power 
Volvo Cars World First 
Collision Warning With 
Auto Brake System 

> Volvo: New Collision 
Warning with Auto Brake 
Helps Prevent Rear-end 



Events 


EyeQ 


Vision on 
a Chip 



> Vision Applications 



Road, Vehicle, 
Pedestrian Protection 
and more 


> read more 


■ AWS Advance 

Warning System 



j 


> Hobdeye at Equip Auto, 

Paris, France 


> Mobile ve at 5EMA, Las 

Vegas, NV 


> read more 


• Mobileve 

• Vision systems currently in high-end BMW, GM, Volvo models 

• By 2010: 70% of car manufacturers. 




CS 4495 Computer Vision - A. Bobick 


Introduction 


Smart cars are here! 



Nevada embraces the future, approves 
seif-driving cars 

1 Coni ment 


: Bill : ■ = = . .5 :: z -s : i co sc 


. - Ir- J 
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Vision-based interaction (and games) 


(* 



Nintendo Wii has camera-based IR 
tracking built in. 


Steve Seitz 
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But the game changer: 
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NASA'S Mars Exploration Rover Spirit captured this westward view from atop 
a low plateau where Spirit spent the closing months of 2007. 

Vision systems (JPL) used for several tasks 

• Panorama stitching 

• 3D terrain modeling 

• Obstacle detection, position tracking 

• For more, read “ Computer Vision on Mars ” by Matthies et al. 

Steve Seitz 
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Robotics 



NASA’s Mars Spirit Rover 
http://en.wikipedia.org/wiki/Spirit rover 


Introduction 



http://www.robocup.org/ 


Steve Seitz 
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Medical imaging 


3D imaging 
MRI, CT 


Introduction 



Image guided surgery 
Grimson et al., MIT 


Steve Seitz 
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Current state of the art 

• You just saw examples of current systems. 

• Many of these are less than 5 years old 

• This is a very active research area, and rapidly changing 

• Many new apps in the next 5 years 

• To learn more about vision applications and companies 

• David Lowe maintains an excellent overview of vision companies 
• http://www.cs.ubc.ca/spider/lowe/vision.html 


Steve Seitz 
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Why is this hard? 
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Simple scene right 


Dark square 


Light square 



Edward H. Adelson 


Introduction 
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Really? 


Edward H. Adelson 


Introduction 
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Vision is NOT Image Processing 

• In the previous example, the two squares have exactly the 
same measurement of intensity. 

• So, seeing is not the same as measuring properties in the 
image. 

• Rather, “seeing” is building a percept of what is in the 
world based upon the measurements made by an imaging 
sensor. 
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Building models from change (1) 



VI 


A»i V 








...I 


Michael Black 





CS 4495 Computer Vision - A. Bobick 


Introduction 


Building models from change (1) 



Left Image 


Michael Black 
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Building models from change (1) 



Right Image 


Michael Black 
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Dan Kersten, http://vision.psych.umn.edu/nsers/kersten/kersten-lab/shadows.html 
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Building models from change (3) 



Dan Kersten, http://vision.psych.umn.edu/users/kersten/kersten-lab/shadows.html 
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Going forward 

• For coming lectures: 

• FP: Chapter 4 (all), 5.1, 5.2 

• Get yourself Matlab (and/or Python/OpenCV) 

• PSO - yes it’s easy but you need to do it and 
need to submit it in the format requested. 



